Visualizing the details of different cellular structures is of great importance to elucidate cellular functions. However, it is challenging to obtain high quality images of different structures directly due to complex cellular environments. Fluorescence microscopy is a popular technique to label different structures but has several drawbacks. In particular, labeling is time consuming and may affect cell morphology, and simultaneous labels are inherently limited. This raises the need of building computational models to learn relationships between unlabeled and labeled fluorescence images, and to infer fluorescent labels of other unlabeled fluorescence images. We propose to develop a novel deep model for fluorescence image prediction. We first propose a novel network layer, known as the global transformer layer, that fuses global information from inputs effectively. The proposed global transformer layer can generate outputs with arbitrary dimensions, and can be employed for all the regular, down-sampling, and up-sampling operators. We then incorporate our proposed global transformer layers and dense blocks to build an U-Net like network. We believe such a design can promote feature reusing between layers. In addition, we propose a multi-scale input strategy to encourage networks to capture features at different scales. We conduct evaluations across various label-free prediction tasks to demonstrate the effectiveness of our approach. Both quantitative and qualitative results show that our method outperforms the state-of-the-art approach significantly. It is also shown that our proposed global transformer layer is useful to improve the fluorescence image prediction results.

An analysis of the paper In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images published in Cell. Take a look at this image, and tell me what you see. If you're a decently smart human being with some background in biology (or not), you probably guessed "some cells?" To be more specific, this a human motor neuron culture derived from induced pluripotent stem cells. If you somehow managed to guess that, let me ask you some more questions.

The Allen Institute for Cell Science has launched the first predictive and comprehensive, 3D model of a live human cell, the Allen Integrated Cell. By allowing researchers around the world to see many structures inside a living cell together at the same time, the Allen Integrated Cell provides a baseline for understanding cells and studying human disease models. "This is a new way to see inside living human cells," said Rick Horwitz, Ph.D., Executive Director of the Allen Institute for Cell Science. In the future, this will impact drug discovery, disease research and how we frame basic studies involving human cells." "Nearly every biologist has mental models of whole cells that are pieced together over their careers with information from dozens of different types of data," said Graham Johnson, Ph.D., Director of the Animated Cell. "The Allen Integrated Cell provides a new option, where 3-D visualizations of whole living cells link to analysis tools to allow for more direct data-driven exploration and hypothesis generation." The Allen Integrated Cell summarizes a large collection of live human cells, gene edited by Allen Institute scientists to incorporate fluorescent protein tags. These tags illuminate specific structures inside of cells, such as the nucleus and mitochondria. Scientists took pictures of tens of thousands of these glowing cells and used artificial intelligence to study them. First, researchers developed a computer algorithm that studied the shape of the plasma membrane, the nucleus and other fluorescently labeled cell structures to learn their spatial relationships. A powerful probabilistic model emerged from this training, that accurately predicts the most probable shape and location of structures in any cell, based solely on the shape of the plasma membrane and the nucleus. Second, researchers took images of those same fluorescently labeled cells and applied a different machine learning algorithm. This algorithm used what it learned from cells with fluorescent labels to find cellular structures in cells without fluorescent labels. This label-free model can be used on relatively easy to collect brightfield microscope images to visualize the integration of many structures inside of cells, simultaneously and with high precision. Viewed side-by-side, the images generated by the label-free method look nearly identical to the fluorescently labeled photographs of cells. "Fluorescence microscopy is expensive and toxic to cells; increasingly so when you tag multiple structures," said Molly Maleckar, Ph.D., Director of Modeling. "Our approach allows scientists to view cells and conduct experiments at the reduced cost of brightfield microscopy, with the structure-identifying power of fluorescence microscopy - and without its toxic effects.

Over the course of their careers, biologists develop a huge mental library of cell structures and their corresponding data. Investigating specific areas of a living cell involves a piecemeal approach, identifying how some parts work with others and spending time on cell labelling. But now, the Allen Institute for Cell Science has launched the first predictive 3D model of a live human cell -- the Allen Integrated Cell -- and it could be "a total game changer", according to researchers. "This is a new way to see inside living human cells," said Rick Horwitz, executive director of the Allen Institute for Cell Science. In the future, this will impact drug discovery, disease research and how we frame basic studies involving human cells." To create the breakthrough model, scientists gene edited a large collection of live human cells to incorporate fluorescent protein tags, which illuminate specific structures inside the cells. The team took tens of thousands of pictures of these glowing cells, and used AI to create a probabilistic model, which predicts the most probable shape and location of structures in any cell, based on the shape of the plasma membrane and the nucleus. The team then applied a different machine-learning algorithm to those pictures, which used what it learned from cells with fluorescent labels to find cellular structures in cells without fluorescent labels. By combining all the data, the system was able to generate images that look nearly identical to those obtained by traditional fluorescence microscopy, which can be expensive and toxic. "Until now, our ability to see what is going on inside of human cells has been very limited," said Michael Elowitz, professor of biology, bioengineering and applied physics at California Institute of Technology. "Previously, we could only see the proteins that we deliberately labeled.

It's harder than you might think to look at a microscope image of an untreated cell and identify its features. To make cell characteristics visible to the human eye, scientists normally have to use chemicals that can kill the very cells they want to look at. A groundbreaking study shows that computers can see details in images without using these invasive techniques. They can examine cells that haven't been treated and find a wealth of data that scientists can't detect on their own. In fact, images contain much more information than was ever thought possible.

Summary Microscopy is a central method in life sciences. Many popular methods, such as antibody labeling, are used to add physical fluorescent labels to specific cellular constituents. However, these approaches have significant drawbacks, including inconsistency; limitations in the number of simultaneous labels because of spectral overlap; and necessary perturbations of the experiment, such as fixing the cells, to generate the measurement. Here, we show that a computational machine-learning approach, which we call "in silico labeling" (ISL), reliably predicts some fluorescent labels from transmitted-light images of unlabeled fixed or live biological samples. ISL predicts a range of labels, such as those for nuclei, cell type (e.g., neural), and cell state (e.g., cell death). Because prediction happens in silico, the method is consistent, is not limited by spectral overlap, and does not disturb the experiment.